THE CENTER ON 
WS “TANI ADI 


WestEd®. CRESST 


Peer Review Assistance 


The U.S. Department of Education's review of State assessment systems is an evidence-based, peer review 
process for which each State submits evidence to demonstrate that its assessment system meets a set of 
established criteria, called critical elements. Critical Elements represent the ESEA statutory and regulatory 
requirements that State assessment systems must meet. The six sections of critical elements that cover 
these requirements are: (1) Statewide System of Standards and Assessments, (2) Assessment System 
Operations, (3) Technical Quality — Validity, (4) Technical Quality —- Other, (5) Inclusion of All Students, and (6) 
Academic Achievement Standards and Reporting. 


Each State must submit evidence for its assessment system that addresses each critical element. The Department's 
assessment peer review focuses on the processes for assessment development employed by the State and the 
resulting evidence that confirms the technical quality of the State’s assessment system. Based on the evidence a State 
submits, the reviewers evaluate the State’s assessment system against ESEA requirements. Per the non-regulatory 
guidance issued by USED, the assessment peer review notes serve as both the “record of the assessment peer review 
team’s evaluation of a State’s evidence” and are sent to the State as “technical assistance and preliminary feedback 
prior to a formal decision regarding the outcome of the review”. “The assessment peer review notes, however, do not 
necessarily identify the final set of additional evidence, if any, that a State may need to submit to demonstrate that its 
assessment system meets all of the critical elements.” A State should consider such feedback as technical assistance 
and not as formal feedback or direction to make changes to its assessment system. Instead, for information regarding 
the adequacy of submitted evidence, states should refer to the formal feedback received from the USED Assistant 
Secretary of Elementary and Secondary Education (formal feedback is submitted to states and can be found in publicly 
published decision letters, while the peer review notes are sent directly to states and are not publicly posted). 


Formal feedback is provided to a State regarding whether the State has provided sufficient evidence to demonstrate 
that its assessment system meets all applicable ESEA statutory and regulatory requirements following the assessment 
peer review. If a State has not provided sufficient evidence, the formal feedback will identify the additional evidence 
necessary to address the critical elements. Per the non-regulatory guidance issued by USED, “The Department will 
work with the State to develop a plan and timeline for submitting the additional evidence for assessment peer review.” 
To support states and ensure that the correct evidence is submitted for each critical element, CSAI can provide 
technical assistance supporting the different requirements. 


The data organizers on the following pages highlight critical elements of peer review, specifically, those critical 
elements for which the Center on Standards and Assessment Implementation (CSAI) can provide support and/or 
resources. 


a Peer Review Assistance 


1 Statewide System of Standards and Assessments 


CRITICAL ELEMENT. CSAI OFFERED SUPPORTS AND RESOURCES 


1.1 = Recommendations for and insight into appropriate documentation 
State Adoption of for demonstrating evidence of alignment between states’ content and 
Academic Content — achievement standards. 

Standards for All = Support in gauging the rigor of state adopted academic content 
Students standards. 


National Benchmarks for State Achievement Standards 


le. = Assistance determining the alignment and rigor of state content 
Coherent and Rigorous standards, as compared to other sets of standards or expectations. 
Academic Content = Support for determining the rigor, coherence, and expectations 
Standards embedded within state content standards. 


= Addressing the need for a broad and diverse stakeholder involvement in 
the development of state content standards. 


States’ Content Standards Revision Processes | Web Alignment Tool 


1.3 = Provision of sample documentation that outlines state inclusion 

Required Assessments = of all required assessments, including annual general and alternate 
assessments in reading/language arts and mathematics (in grades 3-8 
and at least once in high school) and science (at least once in each of 
three grade spans: 3-5, 6-9, and 10-12). 


State of the States Testing Action Plan Spotlight 


1.4 = Support for state plans and policies around accommodations for 

Policies for Including all students. 

All Students in = ™ Assistance in crafting policies that ensure students of special 

Assessments populations have the needed supports to participate in state 
assessments. 


= Clarify different options that states have for including newly arrived 
English learners in state assessment and accountability systems. 


Principles and Characteristics of Inclusive Assessment Systems ina 
Changing Assessment Landscape 


1.5 


Participation Data 
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) Assessment System Operations 


CRITICAL ELEMENT. CSAI OFFERED SUPPORTS AND RESOURCES 


Dall = Assistance in demonstration of alignment between assessment 
Test Design and purposes and data use, as well as between assessment items and 
Development content standards. 


= Support for assessment blueprint analysis and addressing identified 
gaps and weaknesses in the alignment between assessment items and 
content standards. 


Alignment and Evaluation of College and Career Ready Standards 
and Assessments 


Evaluating Content Alignment in the Context of Computer-Adaptive 
Testing: Guidance for State Education Agencies 


DD = Guidance on best practices for item development, item exemplars, or 
Item Development —— feedback on item development process. 


Item Writing Guides and Item Exemplars (ELA & Math) 
Science Assessment Item Collaborative 


D283 = Support in the creation of clear and effective communication tools 
Test Administration = that outline policies and procedures related to test administration. 


Communications 101: Getting Your Message Out 
Helping SEAs Communicate Effectively Around Shifts in Assessment 


Scores 
2.6 = Guidance on the establishment of minimum n size for accountability 
Systems for Protecting .. ‘eterminations. 
Data Integrity Privacy = Provide examples of other states’ policies and procedures regarding the 


confidentiality of test materials, test results, and students’ identifiable 
information, supporting creation of state policies and procedures that 
are consistent with FERPA and applicable state laws. 


2.5 = Examples of state policies and procedures regarding security and the 
Test Security —— prevention of test irregularities. 
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Technical Quality — Validity 


‘CRITICAL ELEMENT CSAI OFFERED SUPPORTS AND RESOURCES 


3.1 m Leverage extensive experience establishing and documenting content 
Overall Validity, aa validity on behalf of states for the purposes of peer review compliance. 
including Validity = Provide examples of the documentation states produce to demonstrate 
Based on Content that assessments are cognitively appropriate for students at each grade 
3.2 level. 

Validity Based on = Provide examples of the documentation states produce to demonstrate 
Cognitive Processes that assessments are consistent with sub-domain structures of state 


content standards. 
3.03 
Validity Based on 
Internal Structure 


= Provide examples of the documentation states produce to demonstrate 
that assessments’ external validity matches with that of other variables, 
including those of other assessments. 


3.4 
Validity Based on Item Writing Guides and Item Exemplars (ELA & Math) 
Relationships with Science Assessment Item Collaborative 


Other Variables 
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—A | Technical Quality - Other 


CRITICAL ELEMENT. CSAI OFFERED SUPPORTS AND RESOURCES 


4.2 = Communication support for states as they outline and detail the steps 
Fairness and taken to ensure that assessments are accessible and fair for all students, 
Accessibility without fundamentally altering assessment constructs. 


= Resources that guide the development and implementation of an 
assessment system that is inclusive and accessible for all students, 
including how to document procedures for identifying potential bias in 
assessment items. 


Fairness in Assessment of English Learners 


A State Guide to the Development of Universally Designed Assessments 


4.1 = Support in determining which pieces of evidence to submit and/or 
Reliability — create for demonstration of assessment reliability. 


= Insight into and examples of what states may use in their peer review 
documentation to demonstrate criterion-related validity evidence, inter- 
rater reliability, structural consistency, and scoring and analysis for all 
assessment types and forms. 


Alignment and Evaluation of College and Career Ready Standards and 
Assessments 


Evaluating Content Alignment in the Context of Computer-Adaptive 
Testing: Guidance for State Education Agencies 


4.4 = States can draw upon CSAI leaders’ prior work in ensuring that 
Scoring — assessments are reliable and produce valid score interpretations. 
4.5 = Support to ensure that all assessment options are properly aligned to 


Multiple Assessment =~ state content standards. 


Forms = Support regarding efforts to compare score interpretations to gauge the 
level of comparability between across different assessments. 


4.6 = Examples of documentation outlining how to compare score 

Multiple Versions of an — interpretations across different assessments. 

Assessment = Guidance on demonstrating alignment between statewide 
assessments and alternate assessments/assessments in languages other 
than English. 
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5 Inclusion for All Students 


5.1 

Procedures for 
Including Students 
with Disabilities 


5.2 
Procedures for 
Including ELs 


5.3 


Accommodations 


5.4 

Monitoring Test 
Administration for 
Special Populations 


CSAI OFFERED SUPPORTS AND RESOURCES 


Guidance on policies and practices that ensure students with 
disabilities are fully included in a state’s assessment system, including 
information on appropriate and needed assessment accommodations. 


Support for demonstrating alignment and coherence between 
academic standards and alternate achievement standards. 


Students with Disabilities 


Assistive Technologies for Computer-Based Assessments 


Student Assessment Inventory for School Districts: Considerations for 
Special Education Assessment Systems 


Resources that guide policies and practices that ensure English learner 
students are fully included in a state’s assessment system, including 
information on appropriate and needed assessment accommodations. 


Guidance on how newly arrived English students are to be included in 
states’ assessment and accountability systems. 


English Language Learners 


Student Assessment Inventory for School Districts: Considerations for 
Assessing English Language Learner Students 


Examples of appropriate assessment accommodations that states can 
consider for implementation into their assessment system. 


Accessibility Review Policies for Accessibility and Accommodations 


Principles and Characteristics of Inclusive Assessment Systems ina 
Changing Assessment Landscape 


Assistance in documenting plans and policies to monitor students who 
receive assessment accommodations to make sure accommodations 
are appropriate for students’ needs and consistent with the 
accommodations provided to students during instruction. 
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Academic Achievement Standards and Reporting 


CRITICAL ELEMENT CSAI OFFERED SUPPORTS AND RESOURCES 


6.1 = Support for gauging the rigor of state achievement standards to 

State Adoption = ensure that states have formally adopted challenging academic 

of Academic achievement standards for all students in each grade and content area, 

Achievement including alternate academic standards. 

Standards for All 

Students 

6.2 = Support in developing technically sound standards setting processes. 

Achievement ™ Guidance in the documentation of the achievement standards setting 

Standards-Setting process and its descriptors for each grade and content area. 

6.3 = Alignment support for states looking to ensure that achievement 

Challenging and 4 standards are aligned with content standards, including conducting 

Aligned Academic external and impartial alignment studies. 

Achievement = Support of state efforts to document how cut scores are determined to 

Standards delineate performance levels, providing students with understanding of 
how each achievement level corresponds with performance. 
Achievement Level Setting 

6.4 = Guidance in developing communication tools that allow states to 

Reporting — effectively communicate results for all assessments and the meaning 


of scores to various audiences (students, teachers, school staff, 
community members). 


= Support to properly document processes for creating data reports, 
ensuring that reports are disseminated in a timely manner and 
that reports include necessary score analyses and descriptions and 
performance levels. 


Communications 101: Getting Your Message Out 
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promote excellence, achieve equity, and improve learning for children, youth, and 
adults, WestEd has more than a dozen offices nationwide, from Massachusetts, Vermont 


Planning for the Every Student Succeeds Act (ESSA): Standards is produced by the Center on Standards and Assessment and Georgia, to Illinois, Arizona and California, with headquarters in San Francisco. For 
Implementation (CSAl). CSAI, a collaboration between WestEd and CRESST, provides state education agencies (SEAs) and Regional more information about WestEd, visit WestEd.org; call 415.565.3000 or, toll-free, (877) 
Comprehensive Centers (RCCs) with research support, technical assistance, tools, and other resources to help inform decisions about 4-WestEd; or write: WestEd / 730 Harrison Street / San Francisco, CA 94107-1242. 
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